In the three previous experiments, we have shown that people infer trustworthiness from convergence - the extent to which different informants agree. Greater convergence makes (i) an information to be perceived as more accurate and (ii) the informants as more competent - that is, given certain context.
Some aspects of that context we held constant; others we varied to evaluate how they alter the effect of convergence. We held constant that participants did not have any priors whatsoever - not on the information they received (they knew nothing about the games that were played) nor about the informants (the players who played the games). We manipulated informants’ independence. We tested two cases of independence, by varying the baseline of what dependent meant: In experiment two, dependent informants had discussed “at great length” before making their estimates. In experiment three, dependent informants shared the same conflict of interest of making a certain estimate.
In this first series of experiments (experiments one, two and three) we tested inferences from convergence in a numerical choice setting: Participants saw (fictive) players’ estimates that would lie with 1000 and 2000. The degree of convergence varied by the distance between estimates.
In a second series of experiments (experiments four, five and six), we test inferences from convergence in a categorical choice setting. In the categorical scenario, the fictive players make choices on a set of response options (i.e. categories). Convergence varies by the ratio of people agreeing on an option. Experiment four and five can be considered robustness checks as to whether the results of the first series hold in a categorical choice setting. Experiment six tests a new context factor: the number of choice options. In experiments four and five, scenarios always involved three choice options. Here, we will vary between three and ten options. The design we use to manipulate convergence is otherwise identical to experiment four (players playing games).
No data has been collected yet.
Participants see eight situtations in which three advisors give investment recommendations. They are then asked to judge one advisor’s accuracy and competence. We will manipulate two factors: convergence (four levels, within participants) and options (two levels, between participants).
Introduction for participants: “To be able to understand the task, please read the following instructions carefully: Some people are playing games in which they have to select the correct answer among three answers. You will see the results of several of these games. Each game is different, with different solutions and involving different players. All players answer independently of each other. At first, you have no idea how competent each individual player is: they might be completely at chance, or be very good at the task. It’s also possible that some players are really good while others are really bad. Some games might be difficult while others are easy. Your task will be to evaluate the performance of one of the players based on what everyone’s answers are.”
An example of a stimulus for the majority condition
Convergence. Convergence varies by the ratio of
players choosing the same response as the focal player (i.e. the one
that participants evaluate). The levels of convergence are: (i)
consensus, where all three players pick the same option
[coded value = 3]; (ii) majority, where either the third or
second player picks the same option as the first player
[coded value = 2]; (iii) dissensus, where all three players
pick different options [coded value = 1]; (iv) majority
against the focal player’s estimate, where the second and third player
pick the same option, but one that is different from the first player’s
choice [coded value = 0]. In our analysis, we treat
convergence as a continuous variable, assigning the values in squared
parenthesis.
We manipulate convergence within participants. All participants see all four conditions of convergence, with two stimuli (i.e. game results) per condition. Each participant therefore sees eight stimuli in total (4 convergence levels x 2 stimuli) .
| Level | Version a) | Version b) |
|---|---|---|
| opposing majority (0) | ||
| dissensus (1) | ||
| majority (2) | ||
| consensus (3) |
Options. Independence has two levels: (i) ‘3’ and (ii) ‘10’.
We manipulate independence between participants, i.e. each participant gets assigned to either the ‘3’ or the ‘10’ condition. All stimuli that participants see in their respective condition will involve the same number of choice options.
| Convergence | Options: 3 | Options: 10 |
|---|---|---|
| consensus (3) |
As outcome variables, we will measure people’s perceived accuracy and competence of player one.
Accuracy. We ask participants “What do you think is the probability of player 1 being correct?”. Participants answer with a slider from 0 to 100.
Competence. We ask participants “How competent do you think player 1 is in games like these?” Participants answer on a 7-point Likert scale (from “not competent at all” to “extremely competent”).
Considering only 3 options condition of the experiment
at hand, we expect the findings of experiment four to replicate. In
experiment four, players would choose among 3 options.
To test this hypothesis, we only consider participants assigned to
the 3 options condition.
We use a linear mixed effect model with random intercept and random slope per participant. Should this model yield convergence issues, we will use a model with random intercept only.
In all our models we treat convergence as a continuous
variable. We will, however, include robustness checks where we treat
convergence as a categorical variable, allowing to inspect difference
between different levels.
# models for accuracy
# random intercept and slope by participants
model_accuracy <- lmer(accuracy ~ convergence + (1 + convergence | id),
data = data %>% filter(options == "3"))
# in case of non-convergence: random intercept by participants only
alt_model_accuracy <- lmer(accuracy ~ convergence + (1 | id),
data = data %>% filter(options == "3"))
To test this hypothesis, we only consider participants assigned to
the 3 options condition.
We will proceed in the same way for competence as we did
for accuracy above.
# models for competence
# random intercept and slope by participants
model_competence <- lmer(competence ~ convergence +
(1 + convergence | id),
data = data %>% filter(options == "3"))
# in case of non-convergence: random intercept by participants only
alt_model_competence <- lmer(competence ~ convergence + (1 | id),
data = data %>% filter(options == "3"))
How about a context in which informants (i.e. the fictive players) could choose not among 3, but among 10 options? In line with our findings from experiment three, we predict that effects of convergence are more positive when there are more choice options:
To test this hypothesis, we consider the full data.
The resulting estimate of the interaction term will provide the test for our hypothesis.
# models for accuracy
# random intercept and slope by participants
model_accuracy <- lmer(accuracy ~ convergence + options +
options*convergence + (1 + convergence | id),
data = data)
# in case of non-convergence: random intercept by participants only
alt_model_accuracy <- lmer(accuracy ~ convergence + options +
options*convergence + (1 | id),
data = data)
To test this hypothesis, we consider the full data.
The resulting estimate of the interaction term will provide the test for our hypothesis.
# models for competence
# random intercept and slope by participants
model_competence <- lmer(competence ~ convergence + options +
options*convergence + (1 + convergence | id),
data = data)
# in case of non-convergence: random intercept by participants only
alt_model_competence <- lmer(competence ~ convergence + options +
options*convergence + (1 | id),
data = data)
In the models above, we treated convergence as a continuous variable.
Based on the different levels, we will build a categorical variable,
convergence_categorical.
# make a categorical variable from `convergence`
data <- data %>%
mutate(convergence_categorical = recode_factor(convergence,
`0` = "opposing majority",
`1` = "divergence",
`2` = "majority",
`3` = "consensus",
.default = NA_character_)
)
levels(data$convergence_categorical)
## [1] "opposing majority" "divergence" "majority"
## [4] "consensus"
We run the same models outlined in the hypotheses section, but
replacing convergence with
convergence_categorical. This also allows us to inspect
heterogeneity in differences between levels (with respect to the
baseline, i.e. “opposing majority”).
We kept the same distance between 10 and 3. That’s conservative - people might think that other options are not relevant and hence don’t make a difference between 3 and 10 options. We therefore run the hypotheses set two comparing 3 to a version of 10 with larger distance.
Pick an example stimulus as earlier for illustrating choice options. See all stimuli in appendix.
We will exclude participants failing (i.e. participants not answering the question or writing anything that does not at least resemble “I pay attention”) the following attention check:
Imagine you are playing video games with a friend and at some point your friend says: “I don’t want to play this game anymore! To make sure that you read the instructions, please write the three following words”I pay attention” in the box below. I really dislike this game, it’s the most overrated game ever. Do you agree with your friend?
We ran a power simulation to inform our choice of sample size. All
assumptions and details on the procedure can be found in the
power_Exp5.Rmd document. We ran two different power
analyses, one for each outcome variable. We set the power threshold for
our experiment to 90%.
The power simulation for accuracy suggested that for 80
participants, we would have a power of at least 90% for the interaction
effect. The simulation for competence suggested that with
already 40 participants, we would detect an interaction, but only with
60 participants we also detect an effect of convergence.
However, due to uncertainty about our assumptions and because we are
anticipating failed attention checks, we will recruit a sample of
200 participants.
| Level | Version a) | Version b) |
|---|---|---|
| opposing majority (0) | ||
| dissensus (1) | ||
| majority (2) | ||
| consensus (10) |
| Level | Version a) | Version b) |
|---|---|---|
| opposing majority (0) | ||
| dissensus (1) | ||
| majority (2) | ||
| consensus (10) |